TITLE OF THE INVENTION 



APPARATUS AND METHOD FOR CONTROLLING 
A CAMERA BASED ON A DISPLAYED IMAGE 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to a camera control 
system, and to a camera control method suitable for use with 
a system in which information communication is performed 
through a network, for example, a monitoring camera system 
or a television conference system. 
Description of the Related Art 

Conventionally, monitoring systems employing cameras 
have been used in comparatively small buildings. In such 
monitoring systems, a certain number of cameras are operated 
to form images of objects to be observed and the images in 
accordance with analog image signals output from the cameras 
are displayed on a plurality of monitor screens. 

A camera control system has been proposed which can 
connect a plurality of cameras of a monitoring system of the 
above -de scribed kind to a digital network such as a local 
area network (LAN) or to a public digital network such as an 
integrated service digital network (ISDN) , and which enables 



a larger number of cameras in remoter places to be easily- 
connected to the monitoring center. 

For use in such a camera control system, a system has 
also been proposed in which a personal computer or a 
workstation is used as a camera operating terminal to 
perform camera control and image display with camera 
operations using a graphical user interface (GUI) . 

Such a graphical system has the advantage of enabling 
even a person unfamiliar with the control system to easily 
execute desired control operations. In particular, the 
facility of control can be improved by the method of 
displaying on a monitor screen a control panel of a camera 
along with the image output from the camera. 

The method of displaying a control panel of a camera 
along with the image output from the camera on the monitor 
screen, however, entails a problem of a need to reduce the 
size of the image from the camera so as to leave a space for 
display of the camera control panel, as shown in Fig. 21, or 
a need to display the control panel so that the control 
panel overlaps the displayed image. 

SUMMARY OF THE I3WENTI0N 
In view of the above -described problem, an object of 
the present invention is to provide a camera control system 
enabling an operator to intuitively control a camera while a 
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large-area display of the image formed by the camera is 
provided . 

To achieve the above -described object, according to one 
aspect of the present invention, there is provided a camera 
control system comprising display means for displaying an 
image formed by a camera on the basis of an image signal 
output from the camera, detection means for detecting a 
figure described on a display surface on which the image is 
being displayed by the display means, output means for 
outputting a command for controlling the camera on the basis 
of the figure detected by the detection means, control means 
for controlling the camera on the basis of the camera 
control command output from the output means . 

According to another aspect of the present invention, 
there is provided a camera control apparatus comprising 
display means for displaying an image formed by a camera on 
the basis of an image signal output from the camera, 
detection means for detecting a figure described on a 
display surface on which the image is being displayed by the 
display means, and output means for outputting a command for 
controlling the camera on the basis of the figure detected 
by the detection means. 

According to still another aspect of the present 
invention, there is provided a method of controlling a 
camera control system comprising a detection step of 
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detecting a figure described on a display surface on which 
an image formed by a camera is being displayed, and an 
output step of output ting a command for controlling the 
camera on the basis of the figure detected in the detection 
step . 

According to a further aspect of the present invention, 
there is provided a storage medium for storing programs for 
executing operational processings in a camera control 
apparatus, the storage medium being stored with programs for 
executing processings for displaying an image formed by a 
camera on the basis of an image signal output from the 
camera, detecting a figure described on a display surface on 
which the image is being displayed, and outputting a command 
for controlling the camera on the basis of the detected 
figure . 

According to still a further aspect of the present 
invention, a camera control system comprises a monitor for 
displaying an image sensed by a camera, the displayed image 
corresponding to an image signal output from the camera. A 
detector is provided for detecting a gestural sign on a 
display surface on which the image is being displayed by 
said monitor. An interface is also provided for outputting 
a command for controlling the camera on the basis of the 
gestural signal detected by said detector. Finally, a 
controller controls the camera on the basis of the camera 
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control command output from the output means . 

These and other objects, features and advantages of the 
present invention will become apparent from the following 
description and the accompanying drawings . 



BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1^/is a diagram showing the configuration of a 
camera control system; 

/ 

Fig. 2 is a block diagram showing the configuration of 
an image transmission terminal 12; 

Fig. 3 /s a block diagram showing the configuration of 
an image reception terminal 18; 

Fig. 4/is a diagram showing the configuration of the 
camera control system; 

Fig. 5 is a diagram showing an example of the contents 
of a display on the monitor screen of the image reception 
terminal 18 y 

Fig. 6y is a diagram showing an example of the contents 
of a display on the monitor screen of the image reception 
terminal 18; 

Fig. 7 /is a diagram showing an example of the contents 
of a display on the monitor screen of the image reception 
terminal 18 •] 

Fig. 8 is a diagram showing an example of a display of 
a camera icon when the camera icon is dragged; 
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Fig. 9 is a flowchart of the operation of the image 
reception terminal when the image reception terminal outputs 
a camera control command; 

Fig. 10 // is a flowchart of the operation of the image 
transmission terminal when camera control is executed; 

Fig. 1^/is a diagram showing an example of a display on 
the monitor screen of the image reception terminal in the 
watch mode; ^ 

Figs . 12f& to 12p are diagrams showing stroke patterns 
of gestural signs; 

Figs. 13&. and 13^are diagrams showing stroke patterns 
of other gestural signs ; 

Fig. l^A is a diagram showing a stroke pattern of 
another gestural sign; 

Fig. 14B is a diagram showing an image formed by the 
camera after control with the gestural sign shown in Fig. 
14A; 




Fig. 1E}A is a diagram showing a stroke pattern of 



another gestural sign; 

Fig. 15^B is a diagram showing an image formed by the 
camera after control with the gestural sign shown in Fig. 
15A; / 

Fig. 1 ( 6 is a diagram showing a stroke pattern of 



another gestural sign; 
'ig. 17/i 



Fig. 17/ is a diagram showing an example of an image 



display-setting window; 

Fig. 18 ^da'a. flowchart of the processing operation of a 
CPU in the image^reception terminal; 

Fig. 19A/to 19D/are diagrams showing stroke patterns of 
other gestural signs; 

Fig. 20^ // ±s a diagram showing a stroke pattern of 

/ 

another gestural sign; 
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Fig. 2 OB is a diagram showing an image formed by the 
camera after control with the gestural sign shown in Fig. 
2 OA; and / 

Fig. 23l is a diagram showing an example of a 
conventional display window. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 
A preferred embodiment of the present invention will be 
described below in detail with reference to the accompanying 
drawings . 

Fig. 1 schematically shows the configuration of a 
camera control system which represents an embodiment of the 
present invention. Image transmission terminals 12 (12-1 to 
12-n) are connected to a network 10, through which image 
data and camera control information (including status 
information) in a digital form are transmitted. 

To the image transmission terminals 12 (12-1 to 12-n) , 
which function as camera servers, cameras 16 (16-1 to 16-n) 



are connected through camera controllers 14 (14-1 to 14-n) . 
Each of the camera controllers 14 (14-1 to 14-n) controls 
pan, tilt, zoom and focusing of the corresponding one of the 
cameras 16 (16-1 to 16-n) , with which it connects, in 
accordance with a control signal from the corresponding one 
of the image transmission terminals 12 (12-1 to 12-n) . 

The cameras 16 (16-1 to 16-n) are supplied with 
electric power from the camera controllers 14 (14-1 to 14- 
n) . Each of the camera controllers 14 (14-1 to 14-n) 
performs on-off control of the corresponding one of the 
cameras 16 (16-1 to 16-n) in accordance with an external 
control signal. 

To the network 10 are also connected image reception 
terminals 18 (18-1 to 18-m) which receive image information 
sent from the image transmission terminals 12 (12-1 to 12-n) 
into the network 10, and which display images formed by the 
cameras. To the image reception terminals 18 (18-1 to 18- 
m) , which are client terminals, monitors 20 (20-1 to 20-m) 
constituted of bit-map displays, cathode ray tubes (CRTs) or 
the like are connected. 

The network 10, which is not necessarily a wired 
network, may be a wireless network using wireless LAN 
apparatus. In such a case, image reception terminals 18 may 
be portable image reception terminals integral with monitors 
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Each of the image transmission terminals 12 (12-1 to 
12 -n) compresses an image signal output from the 
corresponding one of the cameras 16 (16-1 to 16-n) in 
accordance with a predetermined compression system such as 
H.261, MPEG (Moving Image Experts Group) or Motion JPEG 
(Joint Photographic Experts Group) , and transmits the 
compressed image signal to some of the image reception 
terminals 18 requesting the image or to all the image 
reception terminals 18. Each image reception terminal 18 is 
capable of controlling various parameters of the cameras 16 
(shooting direction, image magnification, focus, aperture 
and so on) as well as power supply on-off control. 

Each image transmission terminal 12 can also be used as 
an image reception terminal if a monitor is connected to it 
and if it is provided with an image expansion unit for 
expanding compressed images. On the other hand, each image 
reception terminal 18 can also be used as an image 
transmission terminal if camera controller 14 and camera 16 
are connected to it and if it is provided with an image 
compression unit. In such case, these terminals are 
provided with read only memories (ROMs) for storing programs 
for executing image transmission and reception processings. 

Fig. 2 is a block diagram schematically showing the 
configuration of each image transmission terminal 12 . The 
image transmission terminal 12 has a central processing unit 
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(CPU) 22 for overall control of the image transmission 
terminal 12, a main storage 24 (random access memory (RAM)), 
a removable external storage 2 6 such as a floppy disk or a 
compact disk read only memory (CD-ROM) , a secondary storage 
28 such as a hard disk, a mouse 30 which functions as a 
pointing device, a keyboard 32, an I/O board 34 to which the 
camera controller is connected, and which transmits or 
receives a camera control signal, and an image capture unit 
3 6 which captures the image output signal from the camera 



The image capture unit 36 of this embodiment has an A/D 
conversion function for converting an analog video signal 
into a digital signal, and an image compression function for 
compressing image information. 

The image transmission terminal 12 also has a video 
board 38 for displaying image information on the screen of a 
monitor 40, a network interface 42, and a system bus 44 
which connects the components 22 to 38, and 42 to each 
other. If the terminal 12 is provided as a unit for image; 
transmission only, the video board 38 and the monitor 40 can 
be removed . 

The thus -arranged image transmission terminal 12 
transmits an image signal to some image reception terminal 
18 in a remote place, receives a camera control signal 
transmitted from the image reception terminal 18, and 
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executes control of pan, tilt and so on of the camera 16. 

Fig. 3 is a block diagram schematically showing the 
configuration of each image reception terminal 18. The 
image reception terminal 18 has a central processing unit 
5 (CPU) for overall control of the image reception terminal 
18, a main storage 124 (RAM), a removable external storage 
126 such as a floppy disk or a CD-ROM, a secondary storage 
128 such as a hard disk, a keyboard 132, a mouse 151 which 
functions as a pointing device, a video board 13 8 for 
ij*§ 1 0 displaying image information on the screen of an image 

3 

f4 information monitor 140, a network interface 142, a 

J5|= compression decoder 144 for expanding compressed image 

p information, and a system bus 146 which connects the 

^ components of the image reception terminal 18 to each other. 

H( 1 5 The image reception terminal 18 also has a pen type 

H= pointing device 130, which is used to input gestural signs 

SJf described below on the screen of the monitor 140. A 

pressure sensor 150 is provided on the display screen of the 
monitor 140. The pressure sensor 150 detects a pressure 
2 0 when the display screen of the monitor 140 is pressed with 
the pen type pointing device 13 0, and transmits an address 
signal corresponding to a detected area to the CPU 122. The 
CPU 122 identifies a gestural sign described with the pen 
type pointing device 130 on the basis of a signal 
2 5 transmitted from the pressure sensor 150. 
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The pressure sensor 150 for detecting contact between 
the pen type pointing device 130 and the display screen is 
formed of transparent materials and the pressure detection 
system of the pressure sensor 150 may be of any type such as 
an electrostatic capacity type, a surface acoustic wave 
type, a resistance film type, or an infrared type. For 
example, if an electrostatic capacity system is used, 
contact with the panel surface can be recognized from a 
change in electrostatic capacity and the position of contact 
can be detected from changes in small ac currents supplied 
to the pressure sensor 150. Information on the state of 
contact, expressed as coordinate data, is supplied to the 
CPU 122 to identify a gestural sign described on the display 
screen . 

The image reception terminal 18 has the same 
configuration as the image transmission terminal 12 shown in 
Fig. 2 except that it does not have the function of 
controlling a camera and taking in an image formed by the 
camera, and that is has a decoder 144 for expanding a 
compressed image and uses different system software. If the 
image transmission terminal 18 has image transmission 
terminal software loaded therein, it can transmit an image 
output signal to some other image reception terminals 18. 

The image reception terminal 18 transmits a camera 
control signal to one of image transmission terminals 12, 



and the image transmission terminal 12 controls the camera 
16 in accordance with the content of the camera control 
signal and sends back information on the present status of 
the controlled camera 16. The image reception terminal 18 
receives image data sent from the image transmission 
terminal 12, processes the data by predetermined kinds of 
processing and displays the image on the display screen of 
the monitor 140 in a real time manner. 

Fig. 4 shows the configuration of the camera control 
system of this embodiment in terms of software. In the 
secondary storage 128 of each image reception terminal 18 
are stored a camera control client (software) 50 for remote- 
controlling each of the cameras 16 when the image reception 
terminal 18 is connected to the corresponding image 
transmission terminal 12 through the network 10, image 
reception software 52 for expanding compressed image data 
from the image transmission terminal 12 and displaying the 
data on the monitor screen, and map management software 54 
for displaying icons representing the arranged positions and 
the present statuses of the cameras 16, and other programs. 

Image reception software 52 is software for management 
of all the cameras 16 connected to the image transmission 
terminals 12 connected to the network 10. It includes fixed 
information and various kinds of changeable information of 
each camera 16 (e.g., a name of the camera, a host name or a 



name of the image transmission terminal 12 to which the 
camera 16 is presently connected, the state of the camera 
with respect to pan, tilt and zoom, possibility or 
impossibility of camera control, and identification of 
5 cameras 16 presently controlled by the terminal using this 
software or identification of cameras 16 from which 
displayed images are supplied) . These kinds of information 
are also supplied to the camera control client 50 and to the 
map management software 54 and are used, for example, when 

j4 1 0 the camera icon display is changed. 

In the secondary storage 28 of each image transmission 

Jg, terminal 12 are stored a camera control server (software) 56 

for controlling the camera 16 with the camera controller 14 

^. according to a request from one of the camera control 

^ 1 5 clients 50 and informing the requester of information on the 

fO 

H : present status of the camera 16 (the state of use of camera 

Eg 16 and the like) , and image transmission software 58 for 

compressing the output image from the camera 16 and 
transmitting the compressed image in a predetermined format 
2 0 to a requester through the network 10. 

Fig. 5 shows an example of the contents of a display on 
the monitor screen of one of the image transmission 
terminals 18. In a map window 60, the layout of a plurality 
of cameras 16 placed in an office, a store or a warehouse is 
2 5 shown. A plurality of maps 60a, 60b, 60c, and 60d can be 



displayed selectively. The number of displayable maps is 
not particularly limited. 

As shown in Fig. 5, tags are attached to the maps 60a 
to 60d. Each of the maps 60a to 60d can be selectively 
displayed in the front by clicking the pen type pointing 
device 130 pointing at the corresponding tag or by moving a 
cursor 80 onto the tag with the mouse 151 and clicking the 
mouse 151 when the cursor is positioned on the tag. In the 
example shown in Fig. 5, the map 60b is displayed in the 
front and camera icons 62a, 62b, 62c, and 62d representing 
disposed cameras are shown in the map 60b. If there is a 
request for changing the display in the map window 60 from 
the map 60b to the map 60c, the pen type pointing device 13 0 
pointing at the tag of the map 60c or the mouse 151 with the 
cursor placed on the tag of the map 60c is clicked, thereby 
displaying the map 60c, as shown in Fig. 6. Camera icons 
62 e and 62 f corresponding to two cameras disposed in the 
mapped area are shown in the map 60c. The camera icons 62a 
to 62 f are displayed so as to face in the directions 
corresponding to the shooting directions of the cameras 16 
represented by the camera icons 62a to 62 f. 

A window 64 is an image display window having image 
display areas 66a to 66f , in which images from the cameras 
16 are shown. The image display window 64 has a display 
mode for showing a plurality of image display areas, as 



-16- 



shown in Fig. 5, and another display mode for observing only 
one of the plurality of images in an enlarged state. In the 
description of this embodiment, the former mode will 
hereinafter be referred to as "views mode", and the latter 
mode as "watch mode" . 

In the camera control system of this embodiment, if 
there is a request for displaying an image formed by one of 
the cameras 16, a sequence of operations comprising 
selecting the icon representing the camera 16 on the 
corresponding map in the map window 60, superposing the icon 
on one of the image display areas 66a to 66f of the image 
display window 64 and releasing the icon in the superposed 
state (drag and drop) is performed. The image formed by the 
camera 16 is thereby displayed in the image display area 
onto which the icon has been dragged and dropped. 

Fig. 7 shows a case where the camera icon 62c is 
dragged and dropped onto the image display area 66c to 
display the image formed by the camera 16 corresponding to 
the camera icon 62c in the image display area 66c . When the 
camera icon 62c is being dragged for this operation, it is 
changed into a shape such as that shown in Fig . 8 to 
indicate the state of being dragged. 

In operational processing in the image reception 
terminal 18 in this situation, the map management software 
54 informs the image reception software 52 of the identity 
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of the camera corresponding to the selected icon. The image 
reception software 52 finds, from this identity, the camera 
direction, the camera name and the name of the host to which 
the camera is connected. The image reception software 52 
informs the camera control client 5.0 and the map management 
software 54 of these information items. The camera control 
client 50 and the map management software 54 execute the 
above-described operations on the basis of this control 
information. 

If the above -described camera icon drag and drop 
operation is executed on one of the image display areas 
where an image is currently being displayed, a warning 
display is made to provide notice to the operator that the 
camera output ting the image will be changed, thereby 
enabling the operator to select or cancel the display of an 
image corresponding to a newly dragged and dropped icon in 
the image display area. 

Fig. 9 is a flowchart of operational processing in one 
of the image reception terminals 18 when the image reception 
terminal 18 outputs a command to control one of the cameras 
16 . The operation of controlling the camera 16 will be 
described with reference to Fig. 9. 

First, if clicking is performed on one of the image 
display areas 66a to 66 f where an image from one of the 
cameras that the operator wishes to control is being 




displayed, then the above -described display windows are 
changed into a watch mode window such as shown in Fig. 11. 

In the watch mode, one image is displayed in an 
enlarged state. At this time, it is possible to select a 
frame rate priority display such that a displayed moving 
image can move smoothly, or an image quality priority 
display at high resolution, which is made by reducing the 
compression rate. Fig. 17 shows an image display setting 
window for such an image display selection. In Fig. 17 are 
illustrated a frame rate priority button 90 and an image ^ 
quality priority button 92. If one of these buttons is 
clicked, the quality of the image displayed in the window 
shown in Fig. 11 is changed. The window shown in Fig. 17 
can be opened by clicking a setting button 83 shown in Fig. 
11. 

In step slOl, after the display has been changed from 
the viewed mode to the watch mode, the camera control client 
50 sends an inquiry signal to the camera control server 56 
of the image transmission terminal 12, to which the camera 
16 corresponding to the image displayed in the selected 
image display area is connected, to inquire whether or not 
the camera 16 is controllable. 

Referring to Fig. 10, which is a flowchart showing 
operational processing in the image transmission terminal 12 
at the time of controlling the camera 16, the camera control 



server 56 of the image transmission terminal 12 receives the 
inquiry signal in step s201 and sends information on the 
present status of the camera 16 (the state of use of camera 
16 and the like) to the image reception software 52 of the 
image reception terminal 18 in step s202. 

The image reception software 52 receives the camera 
status information in step sl02 and then makes a 
determination in step sl03 as to whether camera control with 
respect to pan and tilt or the like of the camera is 
possible from the received information on the present status 
of the camera 16. 

If the result of the determination is that such camera 
control is possible, the image reception software 52 informs 
the camera control client 50 of this result and the camera 
control client 50 transmits a camera control signal to the 
camera control server 56 of the image transmission terminal 
12 . The camera control signal is formed by the operator in 
accordance with an operating method described below. 

The camera control server 56 receives the camera 
control signal in step s203 and controls the operation of 
the camera 16 in accordance with the camera control signal 
in step s2 04. 

If the result of the determination is that camera 
control is impossible, the image reception software 52 
informs the camera control client 50 of this result and the 




camera control client 50 performs the control such that any 
control signal which may be input by the operator to execute 
a certain operation is ineffective. 

In step sl05 # if the operator wishes to stop camera 
5 control, he or she may click a views button 82. The display 
window is thereby changed into watch mode windows such as 
those shown in Fig. 5. Simultaneously, the camera control 
process is terminated. 

The method of controlling one of the cameras 16 forming 

1 0 an image displayed in the enlarged window in the watch mode 
as shown in Fig. 11 in the case where the camera is 
controllable will next be described in detail. 

In this embodiment, in the case where an enlarged image 
JL. is displayed in the watch mode, camera control is executed 

^ 1 5 on the basis of a gestural sign which is described on the 
N 1 ' monitor 140 in a predetermined manner with the pen type 

pointing device 130, and which is detected by the pressure 
sensor 150 and is recognized by the CPU 122. 

Fig. 12A to 12D show stroke patterns of gestural signs 

2 0 described on the monitor 140 with the pen type pointing 

device 13 0 when pan and tilt of one of the cameras 16 
forming an image displayed in the watch mode window as shown 
in Fig. 11 are controlled. 

Fig. 12A shows a gestural sign for panning the camera 
2 5 16 rightward (an operational form in which the pen type 



y 

ass 



pointing device 13 0 describes a line on the monitor 140 from 
left to right) . Fig. 12B shows a gestural sign for panning 
the camera 16 leftward (an operational form in which the pen 
type pointing device 130 describes a line on the monitor 140 
5 from right to left) . Fig. 12C shows a gestural sign for 

tilting the camera 16 so that the camera 16 faces upward (an 
operational form in which the pen type pointing device 13 0 
describes a line on the monitor 140 along a direction from 
the bottom to the top) . Fig. 12D shows a gestural sign for 
1 0 tilting the camera 16 so that the camera 16 faces downward 
(an operational form in which the pen type pointing device 
13 0 describes a line on the monitor 140 along a direction 
from the top to the bottom) . A row of a plurality of arrows 
shown in each of Figs. 12A to 12D represents one gestural 



1 5 sign and indicates the size and the direction of the stroke 



of the gestural sign. 
Kf In this embodiment, the above-described pan and tilt 

control has a parameter variable according to the size 
(amount of stroke) of each kind of gestural sign. Figs. 13A 
2 0 and 13B respectively show gestural signs each for panning 
. the camera 16 rightward (an operational form in which the 
pen type pointing device 13 0 describes a line on the monitor 
140 from left to right) . 

In comparison between Figs. 13A and 13B, the amount of 
2 5 stroke shown in Fig. 13B is about 2/5 of the amount of 



stroke shown in Fig. 13A. The CPU 122 recognizes each of 
the different amounts of stroke and the stroke direction, 
determines the amount of rightward panning movement of the 
camera 16 according to the amount of the stroke, and 
transmits to the image transmission terminal 12 a camera 
control signal formed on the basis of these determination 
results . 

While the pan and tilt control described above with 
reference to Figs. 12A to 12D is performed by describing on 
the monitor 140 a line in the direction corresponding to a 
desired pan or tilt of the camera 16 by using the pen type 
pointing device, the operation of the pen type pointing 
device for pan and tilt control may alternatively be such 
that, as shown in Figs. 19A to 19D, an arrow is described in 
the direction corresponding to a desired pan or tilt. 

If a gestural sign comprising a segment B and an 
arrowhead segment A is formed in such a manner, the CPU 122 
determines the direction of pan or tilt according to the 
direction of arrowhead segment A in relation to the 
direction of segment B of the gestural sign detected by the 
pressure sensor. Figs. 19A to 19D respectively show a 
gestural sign for panning camera 16 rightward, a gestural 
sign for panning camera 16 leftward, a gestural sign for 
tilting camera 16 so that the camera faces upward, and a 
gestural sign for tilting camera 16 so that the camera faces 
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downward. The amount of panning or tilting movement is 
determined from the length of segment B as in the case of 
pan and tilt control using lines described as shown in Figs. 
12A to 12D. The length of arrowhead segment A is not 
5 particularly limited. 

Fig. 14A shows a stroke pattern of a gestural sign 
described on the monitor 140 with the pen type pointing 
device 130 when zoom control of one of the cameras 16 
forming an image displayed in the watch mode window as shown 

D 

yri 10 in Fig. 11 is performed. 

5 

The gestural sign shown in Fig. 14A is formed by 
'% describing a substantially circular shape with the pen type 

j^H pointing device 130, the circular shape surrounding an image 

B area which is to be zoomed in. If a gesture is made as 

N: 1 5 shown in Fig. 14A, the CPU 122 recognizes a circularly 

w; ■•■ 
I - : 

Mf described gestural sign and also recognizes the position at 

fg ; which the gestural sign is formed and the size of the formed 

gestural sign. 

Then, the CPU 122 transmits a control signal to the 
2 0 image transmission terminal 12 to perform panning and 

tilting so that the center of the circle described at a 
position on the image is positioned at the center of the 
window. Simultaneously, the CPU 122 transmits a signal for 
controlling the zoom ratio according to the size of the 
2 5 described circle to the image transmission terminal 12. 



While pan, tilt and zoom control of camera 16 using a 
circularly described gestural sign such as that shown in 
Fig. 14A has been described, a rectangularly described 
gestural sign such as that shown in Fig. 20A may also be 
formed on the screen of the monitor 140 with the pen type 
pointing device 130 so as to surround an image area which is 
selected to be displayed in an enlarged state. 

If such a rectangular gesture is made, the CPU 122 can 
definitely recognize the enlargement target area to 
accurately form commands for controlling pan, tilt and zoom 
of camera 16. 

Fig. 20B shows an image displayed when the 
rectangularly described gestural sign shown in Fig. 20A is 
given to execute pan, tilt and zoom control of camera 16. 

Fig. 15A shows a stroke pattern of a gestural sign 
described on the monitor 140 with the pen type pointing 
device 13 0 when zoom control of one of the cameras 16 
forming an image displayed in the watch mode window as shown 
in Fig. 11 is performed. 

As shown in Fig. 15A, a gesture is made such as to 
describe "X" with the pen type pointing device 130 on an 

area of the image which is to be zoomed out. The CPU 122 
then recognizes the described gestural sign as "X" and also 

recognizes the position at which the gestural sign is formed 
and the size of the formed gestural sign. 
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The CPU 122 transmits a control signal to the image 
transmission terminal 12 to perform panning and tilting so 
that the point of intersection of the two segments forming 
"X" is positioned at the center of the window. 

Simultaneously, the CPU 122 transmits a control signal for 
reducing the zoom ratio according to the size of the 
described "X" to the image transmission terminal 12. In 
this embodiment, if the size of "X" is increased, the zoom 
ratio designated by the control signal is reduced. 

Fig. 16 shows a gestural operation of moving the pen 
type pointing device 13 0 so as to form one loop on the 
screen of the monitor 140. When the CPU 120 recognizes such 
a gesture, it stops controlling the camera 16 forming the 
image displayed on the screen and loses the power of 
controlling the camera 16. 

Fig. 18 is a flowchart of the processing operation of 
the CPU 122 in the case where a gestural sign, such as one 
in those described above, is formed by the operator. 

In step slO, an input gesture is normalized on the 
basis of the signal representing the result of detection 
made by the pressure sensor 150. Next, in step sll, the 
normalized input gesture pattern is collated with gesture 
patterns previously stored in the external storage 126 (with 
respect to processing relating to the gesture and the stroke 
of the gesture) . 




If collation in step sl2 ends in failure, no processing 
is performed thereafter and the process ends. If collation 
in step sl2 ends in success, a control command (for pan, 
tilt, zoom or the like) corresponding to the collated 
gesture pattern is searched for in step sl3 . In step sl4, 
some of various control parameters, which are the amount of 
panning movement, the amount of tilting movement, the zoom 
ratio and so on, is calculated from the size of the gesture 
pattern or the position at which the gesture pattern is 
formed . 

For example, in pan and tilt control of camera 16 with 
respect to gestures such as those shown in Figs. 12A to 12D, 
each of the amount of panning movement and the amount of 
tilting movement is determined from the amount of stroke of 
the corresponding gesture. With respect to the gesture 
shown in Fig. 14A or 15A, the pan and tilt parameters are 
calculated so that the center of the gesture is positioned 
at the center of the window. Further, the zoom ratio is 
calculated from the size of the gesture. 

In step sl5, a control signal for executing control 
using some of the various parameters calculated as described 
above is transmitted to the image transmission terminal 12 
through the network. 

In this embodiment, normalization of an input gesture, 
the collation of the gesture pattern and search for a 



corresponding control command, and camera control in 
accordance with the control command found by searching are 
executed on the image reception terminal 18 side. However, 
the arrangement may alternatively be such that the input 
5 gesture signal is immediately transmitted from the image 

reception terminal 18 to the image transmission terminal 12 
and the above -de scribed operational processing is performed 
in the image transmission terminal 12. 

This operation method can easily be practiced in such a 

□ 

yr| 1 0 manner that a control program for executing the above- 
pi 

0 described operational processing is stored in the secondary 
yg; storage 28 or the external storage 26, and the CPU 22 reads 
Si; out the control program to the main storage 24 and executes 
p t! the camera control processing in accordance with the 

* 1 5 program. 

1 ^ 

H : As described above, the camera 16 corresponding to a 

tSf displayed image can easily be controlled by performing a 

gestural operation on the screen of the monitor 140 with the 
pen type pointing device 13 0. If this camera control is 
2 0 performed, the need for provision of an operation panel for 
camera control on the screen of the monitor 140 is 
eliminated, so that the image formed by the camera can be 
displayed without being obstructed by the operation panel, 
that is, the image display area can be enlarged by the 
2 5 amount corresponding to the operation panel area. 
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According to the above-described embodiment of the 
present invention, the arrangement may alternatively be such 
that a recording medium on which program codes of software 
for realizing the above-described functions is supplied to a 
system or unit, and this system or unit (CPU, MPU or the 
like) reads out the program codes stored in the storage 
medium and executes processings in accordance with the 
program codes . 

In such a case, the program codes themselves realize 
the functions of the embodiment described above, and the 
storage medium storing the program codes constitutes this 
embodiment . 

The storage medium used to supply program codes may be, 
for example, a floppy disk, a hard disk, an optical disk, an 
magneto-optical disk, a compact disk read only memory (CD- 
ROM) , CD-Recordable, a magnetic tape, nonvolatile memory 
card, or a read only memory. 

The method for realizing the functions of the above- 
described embodiment by executing processing in accordance 
with program codes read out by a computer may also comprise 
performing a part or the whole of the actual processing with 
an operating system or the like running on the computer in 
accordance with instructions of the program codes. 

Further, after the program codes read out from the 
recording medium have been written to a feature expansion 
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board inserted in the computer or to a memory provided in a 
feature expansion unit inserted in the computer, a CPU or 
the like provided in the feature expansion board or unit 
performs a part or the whole of the actual processing in 
accordance with instructions of the program codes, thereby- 
realizing the functions of the above-described embodiment. 

If the above -described recording medium is used, the 
program codes corresponding to the above -described 
flowcharts are stored. Briefly, modules necessary for the 
camera control system of the present invention are stored in 
the storage medium. 

According to the above -described embodiment of the 
present invention, at least program codes of a 
"normalization module" for normalizing an input gesture, a 
"collation module" for collating the gesture normalized by 
the normalization module with gesture patterns previously 
stored, and a "parameter calculation module" for calculating 
each of various control parameters from the size of the 
input gesture or the position of the gesture on the screen 
may be stored in the storage medium. 

According to the embodiment of the present invention, 
as described above, it is possible to provide a camera 
control system which makes it possible to easily control 
pan, tilt or zoom of a camera through a network by 
performing a predetermined gestural operation on a display 
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window while an image formed by the camera is being 
displayed in the display window, which requires no camera 
control operation panel on the screen of the monitor since 
such camera control is performed, and which therefore 
enables the image formed by the camera to be displayed by 
being fully enlarged over the monitor screen while enabling 
the desired camera control. 

According to the embodiment of the present invention, 
it is also possible to provide an image receiving apparatus 
in which pan, tilt or zoom of a camera can easily be 
controlled through a network by performing a predetermined 
gestural operation on a display window while an image formed 
by the camera is being displayed in the display window, 
which requires no camera control operation panel on the 
screen of the monitor since such camera control is 
performed, and in which the image formed by the camera can 
therefore be displayed by being fully enlarged over the 
monitor screen while the desired camera control can be 
performed. 

According to the embodiment of the present invention, 
it is also possible to provide a method of controlling an 
image receiving apparatus which makes it possible to easily 
control pan, tilt or zoom of a camera through a network by 
performing a predetermined gestural operation on a display 
while an image formed by the camera is being displayed in 



r 



the display window, which eliminates the need for a camera 
control operation panel on the screen of the monitor by- 
using the arrangement for such camera control, and which 
therefore enables the image formed by the camera to be 
5 displayed by being fully enlarged over the monitor screen 
while enabling the desired camera control. 

According to the embodiment of the present invention, 
it is also possible to provide a storage medium for storing 
a control program which makes it possible to easily control 

O 

,fb 1 0 pan, tilt or zoom of a camera through a network by 



q; 



performing a predetermined gestural operation on a display 
while an image formed by the camera is being displayed in 
the display window, which eliminates the need for a camera 
1_ control operation panel on the screen of the monitor by 

Mr 15 using the arrangement for such camera control, and which 
hv therefore enables the image formed by the camera to be 

displayed by being fully enlarged over the monitor screen 
while enabling the desired camera control. 

While the present invention has been described with 
2 0 respect to what presently is considered to be the preferred 
embodiment, it is to be understood that the invention is not 
limited to the disclosed embodiment. To the contrary, the 
present invention is intended to cover various modifications 
and equivalent arrangements included within the spirit and 
2 5 scope of the appended claims. The scope of the following 




claims is to be accorded the broadest interpretation so as 
to encompass all such modifications and equivalent 
structures and functions. 
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